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L  SYMMETRIC  SIMULATION  AND  ANALYSIS 

The  concept  of  symmetric  analysis  (SA)'  is  to  conduct  simulation  experiments  in 
which  all  initial  conditions  are  the  same  for  the  two  sides,  except  for  one  difference  (or  a 
small,  controlled  number  of  differences)  whose  effect  is  being  investigated.  This  means 
that  the  terrain  is  symmetric,  the  forces  are  identical,  the  objectives  of  the  two  sides  are 
the  same,  and  the  starting  positions  are  the  same — except  for  the  one  selected  difference. 
It  does  not  mean  that  symmetry  is  maintained  once  the  experiment  is  underway. 

The  primary  purpose  of  SA  is  to  reduce  the  number  of  factors  that  may  affect  the 
outcome  of  a  simulation.  Limiting  the  number  of  variables  is  a  standard  analytic  and 
experimental  procedure,  the  goal  of  which  is  to  better  understand  and  quantify  the  effects 
of  the  variable(s)  that  remain.  SA  provides  a  precise  analog  of  a  controlled  experiment 
and,  therefore,  should  be  a  valuable  tool  in  those  instances  in  which  it  may  be  applied 
reasonably.  The  hope  and  the  expectation  is  that  this  benefit  may  prove  to  be  much 
greater  than  is  immediately  apparent.  Terrain  and  mission  asymmetries  do  not  simply 
introduce  single  extraneous  variables,  which  might  be  compensated  for  by  doubling  the 
number  of  runs.  Instead,  they  may  affect  a  battle  or  a  combat  system  in  ways  that  are 
unexpected  and  difficult  to  anticipate  or  understand.  Carefully  controlling  the  terrain  and 
mission  factors  can  allow  analysts  to  isolate  and  understand  both  their  effects  and  the 
effects  of  other  factors  and  to  allow  the  analytic  community  to  build  the  cumulative  body 
of  knowledge  that  characterizes  more  rigorous  disciplines. 

In  addition  to  these  conceptual  advantages,  SA  may  require  considerably  fewer 
runs  to  achieve  the  same  statistical  confidence  and  offers  other  advantages  in  particular 
types  of  analyses. 

The  rest  of  this  chapter  illustrates  potential  uses  of  SA  and  describes  some 
specific  potential  benefits  and  limitations.  Chapter  n  describes  the  terrain  database 
developed  and  used  in  the  project.  Chapter  in  illustrates  the  use  of  SA  in  a  series  of 
simple  experiments,  which  examine  several  aspects  of  the  behavior  of  ModSAF,  a  widely 
used  DIS  simulator  of  computer-generated  forces.  They  are  exploratory  and  illustrate 


'  Symmetric  analysis  was  conceived  originally  by  Frederick  Young. 
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some  ways  in  which  SA  might  be  used.  Although  all  of  the  experiments  involve  ground 
combat,  the  concept  of  SA  applies  to  air  and  naval  engagements. 

A.  SPECIFIC  POTENTIAL  ADVANTAGES 

Some  specific  advantages  of  symmetric  analysis  include  the  potentials  for 
statistical  efficiency,  the  direct  comparing  of  alternative  simulations,  and  several  others 
discussed  below. 

1.  Statistical  Efficiency 

In  some  cases,  symmetric  analysis  may  offer  the  opportunity  to  compare  two 
alternatives  directly  to  each  other,  rather  than  to  compare  each  to  a  common  reference. 
The  advantage  would  be  that  many  fewer  runs  may  be  necessary  in  order  to  obtain 
statistically  valid  results.  The  reasons  for  this  include  the  following: 

•  In  a  symmetric  analysis,  each  run  provides  a  comparison  datum  point  between 
two  alternatives;  in  a  traditional  analysis,  two  mns  (each  alternative  against  a 
standard  enemy)  are  required  for  each  comparison  point. 

•  When  using  separate  runs  comparing  two  alternatives  against  a  standard 
enemy,  the  random  uncertainties  are  additive,  so  that  two  comparison  points 
may  be  needed  to  achieve  the  same  statistical  confidence  as  one  run  that 
directly  compares  the  alternatives. 

Suppose,  for  example,  one  wants  to  compare  two  new  tank  concepts.  The 
traditional  procedure  is  to  make  separate  sets  of  runs  using  each  concept  against  the  same 
opposing  force  and  to  compare  the  results.  Using  symmetric  analysis,  one  set  of  runs 
would  be  made  with  opposing  forces  equipped  with  the  alternative  concepts.  In  each 
case,  the  measure  of  effectiveness  (MOE)  might  be  the  difference  in  the  number  of 
armored  vehicles  lost  by  each  side. 

For  the  traditional  procedure,  the  MOEs  for  each  concept  would  be  averaged  over 
all  runs,  and  the  difference  between  the  averages  computed.  In  order  to  have  confidence 
in  the  result,  enough  runs  would  be  made  to  reduce  the  variance  in  the  resulting  measure 
to  a  statistically  acceptable  level.  Using  the  subscripts  t  for  traditional  methodology,  i  to 
denote  individual  runs  to  be  summed  over,  and  1  and  2  for  the  two-tank  concepts,  and 
assuming  the  same  number  of  runs  (n)  are  made  for  each  concept,  the  measure  of  the 
difference  Dt  between  the  two  concepts  is: 

D,  =  (ZDi.i-ZD2,i)/n 
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The  variance  of  this  statistic  is  Gx  =  (Gi,  ^  +  C2,  h  /  n,  where  Oi,  ^  and  G2.  are  the 
variances  in  the  individual  run  measures  Di,  i  and  D2,  i  for  concepts  1  and  2,  respectively. 

For  the  symmetric  analysis  with  m  runs  (using  subscript  s  to  denote  symmetric 
methodology),  the  measure  of  the  difference  between  the  two  concepts  would  be  Ds  = 

9  9 

EDs,  i  /  m,  with  variance  Gs  =  EGs,  i  /  m. 

9  9 

If  the  variances  Gi  for  all  individual  runs  are  the  same  and  independent,  then  Gt  = 
2Gi‘  /  n  and  =  Gi^  /  m,  so  that  Gt^  would  equal  if  n  equaled  2m.  Since  two  sets  of 
n  runs  are  made  in  the  traditional  case,  four  times  as  many  runs  would  have  to  be  made  to 
achieve  the  same  variance  as  a  symmetric  analysis. 

There  is  no  assurance  that  individual  run  outcome  variances  are  the  same  for 
traditional  and  symmetric  runs,  and  there  are  some  cases  in  which  this  is  unlikely. 
Consider  weapon  alternatives  that  do  not  engage  each  other  directly,  such  as  alternative 
shoulder-fired  antiaircraft  weapons.  The  linkage  between  a  force’s  infantry  defending 
against  enemy  aircraft  and  the  same  force’s  aircraft  attacking  enemy  positions  occupied 
by  infantry  is  indirect  at  best.  A  scenario  that  includes  both  is  likely  to  have  largely 
independent  battles,  similar  to  running  separate  simulations  for  each.  Putting  the  two 
battles  in  one  run  would  result  in  half  the  number  of  runs,  using  twice  the  resources  in 
each  run.  The  MOE  would  be  the  difference  in  the  MOE  for  each  battle,  with  variance 
twice  as  large  as  each  individual  battle.  There  is  no  real  advantage  to  running  separate 
battles  in  the  guise  of  one  symmetrie  simulation. 

For  systems  such  as  tanks  that  interact  directly,  it  would  be  possible  to  use  a 
traditional  asymmetric  scenario  with  two  alternative  concepts  opposing  each  other.  This 
would  still  not  result  in  a  direct  comparison  of  two  concepts,  however.  In  any  set  of  runs, 
it  would  not  be  clear  whether  a  given  result  derived  from  the  tank  characteristics  or 
terrain  and  mission  characteristics.  At  a  minimum,  two  sets  of  runs  would  need  to  be 
made,  with  each  concept  used  on  each  side.  If  the  results  favored  the  same  side  in  each 
set,  the  implication  would  be  that  terrain  or  mission  characteristics  were  causal  factors, 
complicating  the  comparison  of  the  two  concepts. 

2.  Comparison  of  Simulations 

The  degree  of  control  possible  in  symmetric  simulations  may  be  of  particular 
value  in  analyzing  the  performance  of  the  simulation  systems  themselves.  Examples 
within  distributed  interactive  simulation  (DIS)  might  inelude  alternative  semiautomated 
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forces  (SAF)  systems  [e.g..  Modular  SAF  (ModSAF)  vs.  JANUS-Link],  SAP  systems  vs. 
manned  simulators,  or  a  system  against  itself  (ModSAF  vs.  ModSAF  analyses  are 
discussed  in  Chapter  III). 

Training  using  opposing  manned  simulators  in  a  symmetric  setting  might  be 
effective  as  a  competitive  spur.  If  enough  manned  simulators  are  available,  such  training 
could  improve  the  throughput  of  the  training  facility  by  training  two  units  simultaneously. 

While  this  approach  may  be  used  as  a  validation  tool,  it  may  be  more  valuable  as  a 
way  to  gain  understanding  and  further  simulation  development.  For  example,  SAF  vs. 
manned  simulator  exercises  might  be  used  to: 

•  Measure  the  extent  to  which  SAF  behavior  approximates  that  of  manned 
simulators 

•  Identify  the  most  important  types  of  behaviors  that  were  not  simulated  well,  or 
not  simulated  at  all,  by  the  SAF  system 

•  Identify  situations  (e.g.,  types  of  terrain)  in  which  the  SAF  did  not  perform 
well 

•  Characterize  the  relative  performance  of  the  simulators 

•  Calibrate  the  SAF  system  (e.g.,  setting  the  expertise  levels  of  simulated 
commanders). 

For  these  types  of  purposes,  the  realism  constraints  of  symmetric  analysis  may  be 
quite  acceptable.  In  fact,  existing  approaches  to  model  verification  and  validation  rely 
heavily  on  isolating  particular  aspects  of  the  simulation,  such  as  lower  level  performance 
characteristics.  Lower  level  characteristics  also  need  to  be  evaluated  in  a  larger  context  to 
ensure  that  they  collectively  constitute  a  realistic  simulation  (e.g.,  tank  gunner  time  delays 
and  accuracy  need  to  be  part  of  realistic  tank  behavior  during  a  battle).  The  particular 
advantage  of  symmetric  analysis  is  that  it  offers  the  possibility  of  controlling  the  battle 
variables  to  the  extent  that  anomalous  behaviors  may  be  identified  and  isolated  easily. 

3.  Other 

There  may  be  other  advantages  to  using  symmetry  to  support  particular  types  of 
analyses.  Two  that  have  occurred  to  the  authors  are: 

•  Handedness — A  symmetric  terrain  would  offer  a  unique  ability  to  explore 
handedness  (right/left)  biases,  either  in  simulations  or  in  human  operators. 

•  Symmetry  Within  One  Force — One  of  the  experiments  described  in  Chapter  m 
was  a  pincer  attack.  With  symmetric  terrain,  the  scenario  was  designed  so  that 
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pincers  (and  not  the  opposing  forces)  were  symmetric.  There  may  be  other 
cases  in  which  a  symmetry  within  one  force  is  useful. 

B.  REALISM 

Symmetric  battles  do  not  occur,  and  every  symmetric  analysis  must  suffer  some 
lack  of  realism.  However,  all  battle  simulations  suffer  this  lack  to  some  extent.  The  issue 
should  not  be  whether  symmetric  simulation  can  precisely  represent  the  real  world,  but 
whether  it  can  be  used  effectively  to  understand  and  evaluate  battles  and  battlefield 
systems,  especially  human  behavior. 

Opposing  forces  may  be  similar  but,  in  general,  will  have  somewhat  different 
systems  (e.g.,  U.S.  vs.  Russian  tanks).  Clearly,  symmetric  analysis  would  not  be  used  to 
evaluate  weapons  designed  to  operate  against  specific  opposing  characteristics  (e.g., 
designed  to  penetrate  the  armor  of  T-80  tanks).  In  other  contexts,  fighting  against  an 
otherwise  equal  force  may  offer  the  best  test  of  a  system.  In  particular,  command, 

-a 

control,  and  communications  (C  )  systems  are  difficult  to  evaluate,  and  battles  in  which 
different  force  levels  and  weapons  determine  the  outcome  may  not  offer  the  best  means  of 
understanding  effects. 

Constructing  scenarios  with  the  same  mission  for  the  opposing  forces  is  a 
significant  problem.  Four  possibilities  are  offered  here: 

•  Meeting  Engagements — The  mission  might  be  scouting  or  moving  to  a 
location. 

•  Control  of  Terrain  Features — The  objective  would  be  to  gain  control  of  key 
terrain.  The  forces  might  occupy  symmetric  hills  and  be  tasked  with 
occupying  the  other  (while  retaining  control  of  the  original).  Another 
possibility  would  be  to  occupy  and  control  a  hill  located  on  the  symmetry 
seam. 

•  “Trench”  Warfare — The  objective  would  be  to  penetrate  the  opposing  force 
while  retaining  control  of  one’s  own  territory. 

•  “Flag”  Command  Posts  (Chess  Kings) — A  flag  vehicle  would  be  designated 
and  the  objective  would  be  to  destroy  the  opponent’s  flag. 

The  realism  of  symmetric  terrain  is  also  limited.  This  is  discussed  in  Chapter  H. 
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C.  OBSERVATIONS 

Symmetric  analysis  is  not  applicable  when  specific  asymmetric  factors,  such  as 
characteristics  of  enemy  systems,  are  integral  to  the  analysis.  In  other  cases  and  in  the 
long-term  effort  to  build  a  cumulative  body  of  knowledge,  operational  realism  is  less 
important  than  the  ability  to  isolate  and  understand  effects. 

It  is  hard  to  prove,  but  the  authors’  intuition  is  that  symmetric  analysis  will  yield 
different  and,  in  many  ways,  clearer  understanding  of  the  pros  and  cons  of  some  systems. 
It  is  a  new  approach  that  provides  a  different  and  potentially  valuable  view  of  the  choices 
under  consideration.  It  should  be  easier  to  trace  cause  and  effect  in  symmetric  analysis 
than  in  traditional  simulation  setups  in  which  the  results  depend  on  many  differences 
other  than  the  choice  under  consideration.  For  the  example  of  competing  new  tank 
concepts,  direct  confrontation  should  dramatically  highlight  their  relative  strengths  and 
weaknesses,  even  if  either  concept  would  defeat  a  current  enemy. 

Symmetric  analyses  may  prove  of  particular  value  in  man-in-the-loop  experiments 
for  analyzing  command,  control,  communications,  computers,  and  intelligence  (C"*!). 
These  systems  do  not  usually  operate  against  particular  enemy  characteristics  and  systems 
but  should  be  effective  against  any  enemy.  They  are  notoriously  difficult  to  evaluate,  and 
directly  competing  alternative  concepts  may  well  provide  improved  understanding  of 
their  strengths  and  weaknesses. 

It  remains  true  that  a  scenario  may  favor  one  concept  over  another.  The  strengths 
of  one  concept  may  pay  off  more  in  the  chosen  scenario  than  the  strengths  of  the  other. 
However,  understanding  this  scenario  dependence  may  be  easier  in  a  symmetric  setup, 
where  the  strengths  and  weaknesses  of  each  concept  confront  one  another  directly  and  no 
other  differences  exist  between  the  two  sides. 


6 

UNCLASSIFIED 


UNCLASSIFIED 


II.  THE  SYMMETRIC  TERRAIN  DATABASE 

A.  CONCEPT  AND  POTENTIAL  ANOMALIES 

A  key  part  of  symmetric  analysis  is  the  construction  of  symmetric  terrain.  The 
basic  idea  is  to  reflect  an  existing  terrain  area  around  a  boundary  line.  For  digital 
databases  defining  flat  terrain,  this  is  a  conceptually  simple  process.  For  other  terrain,  the 
process  might  be  more  complex,  and  choices  for  reflecting  lines  to  simplify  the  process 
might  be  more  restrictive. 

If  one  reflection  is  good,  are  two  better?  If  a  rectangular  terrain  patch  is  reflected 
around  a  line  (say,  east-west),  the  resulting  area  could  then  be  reflected  around  a 
perpendicular  (north-south)  line.  This  would  create  two  axes  of  symmetry,  so  that  a  force 
could  have  an  internal  left-right  symmetry  in  addition  to  being  symmetric  to  the  opposing 
force.  This  might  be  useful  in  specialized  situations  (perhaps  in  conducting  a  handedness 
analysis)  but  does  not  seem  to  have  any  generalized  advantages.  It  would  be  more 
difficult  to  construct  a  terrain  without  obvious  anomalies.  This  idea  was  not  pursued. 

A  simple  reflection  may  produce  unrealistic  terrain  or  anomalous  features  as  a 
result  of  the  discontinuity  (in  the  derivative)  along  the  reflection  line.  Some  examples 
include: 

•  Sharp  Folds — ^There  will  be  a  sharp  crease  on  the  reflection  line.  Terrain 
sloping  up  to  it  will  have  a  ridge  along  the  line,  but  terrain  sloping  down  will 
have  a  valley.  To  a  large  extent,  these  effects  are  inherent  in  digitized  terrain 
anyway.  As  described  below,  the  terrain  shape  is  defined  by  giving  the 
elevation  at  regularly  spaced  lattice  points,  and  the  crease  along  the  reflecting 
line  is  not  basically  different  from  the  crease  along  any  lattice  line.  However, 
the  reflection  line  would  have  slope  reversals  at  all  points  along  it. 

•  Roads,  Rivers,  and  Other  Curvilinear  Features — Curvilinear  features  crossing 
the  reflection  line  at  an  angle  other  than  perpendicular  will  have  a  sharp  bend, 
producing  various  degrees  of  unrealism.  The  sharpness  of  the  bend  could 
potentially  be  changed  by  manipulating  the  database  to  change  the  path  of  the 
feature,  but  unnatural  shapes  might  persist.  For  example,  if  the  reflection  line 
cuts  across  a  bend  in  a  road,  intersecting  it  twice,  then  (depending  on  which 
part  of  the  road  is  within  the  reflected  area)  the  road  will  either  become  a 
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closed  loop  or  will  become  two  roads  connected  to  the  mirrored  area  but  not 
connected  to  each  other.  Such  behavior  would  be  even  more  unnatural  in  a 
river,  with  the  additional  complication  of  flow  direction. 

•  Terrain  Features  such  as  Hills,  Valleys,  and  Ridges — Many  types  of 
topographic  features  could  not  be  reflected  along  arbitrary  lines  and  appear 
entirely  natural.  In  most  cases,  the  distortion  would  seem  fairly  benign,  e.g., 
two  symmetric  valleys  joining  at  an  angle. 

In  summary,  an  arbitrarily  chosen  reflecting  boundary  is  likely  to  create  a 
symmetric  terrain  database  that  is  quite  unnatural  in  appearance.  However,  the  choice  of 
terrain  for  experiments  on  research  and  development  (R&D)  concepts  and  prototypes  is 
not  usually  severely  constrained.  In  this  case,  it  should  be  possible  to  find  a  terrain  data 
set  that  has  adequate  complexity  but  behaves  properly  near  the  selected  reflecting 
boundary.  Symmetric  analysis  of  air  or  open-ocean  naval  engagements  would 
automatically  have  symmetric  “terrain.” 

B.  THE  DEMONSTRATION  DATABASES 

The  symmetric  terrain  database  is  a  standard  format  S-1000  database  representing 
a  20  km  X  20  km  region  in  which  the  northern  and  southern  halves  are  reflections  of  one 
another.  It  was  produced  by  Durwood  Gafford  of  Lockheed-Martin  (formerly,  Loral 
Aerospace),  Bellevue,  WA  (LADS,  Bellevue).  His  work  was  supported  by  George  Lukes 
of  the  Defense  Advanced  Research  Projects  Agency,  who  liked  the  notion  of  using 
symmetric  terrain  to  conduct  simulation  experiments.  The  effort  began  in  early  spring  of 
1996  under  the  direction  of  Dale  Miller  at  LADS  Bellevue  and  was  completed  by  mid¬ 
summer.  Jeff  Turner  of  the  U.S.  Army  Topographical  Engineering  Center  contributed  in 
an  oversight  role. 

The  database  was  constructed  by  reflecting  a  rectangular  portion  of  an  existing 
terrain  database  across  a  pre-selected  axis.  A  20  km  x  10  km  section  of  the  Hunter- 
Liggett  Reservation  containing  enough  topographical  features  to  sustain  interesting 
engagement  scenarios  was  chosen  as  the  area  to  be  reflected.  This  section  contains  both 
the  Stony  Valley  and  the.  Lower  Stony  Reservoir.  The  southern  boundary  of  this 
rectangle,  the  axis  of  reflection,  intersects  a  series  of  commanding  hills,  which,  arguably, 
might  serve  as  objectives  for  forces  in  a  symmetric  engagement. 

Figure  1,  below,  is  a  schematic  diagram  of  the  terrain  database.  The  MIL  GRIG 
coordinates  are  shown  in  the  diagram.  The  southwest  and  northeast  comers  correspond  to 
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UTM  coordinates  10SFQ5000079500  and  10SFQ7000089500,  respectively.  The 
original  50  km  x  50  km  database  referred  to  in  Figure  1  is  formally  known  as  Hunter  - 
0110.  It  is  shown  in  Figure  2  with  the  reflected  rectangle  highlighted. 

Initially,  in  an  attempt  to  save  development  time,  an  effort  was  undertaken  to 
construct  the  symmetric  terrain  from  a  “run-time”  database.  After  a  few  false  starts,  this 
strategy  was  abandoned,  and  the  more  standard  procedure  of  constructing  a  source  data 
file  using  the  S-1000  Tool  Kit  was  pursued.  This  approach  led  to  the  successful 
completion  of  the  task.  The  resulting  symmetric  terrain  database  is  shown  in  Figure  3. 
Format  ‘4’  and  format  ‘5’  terrain  databases  were  subsequently  compiled  from  the  S-1000 
source  file.  These  formatted  databases  permitted  IDA  to  conduct  symmetric  analyses 
using  ModSAF  versions  2.0  and  2.1. 

Finally,  a  visual,  or  “3-D  out-of-window-view”  database  was  constructed  in 
November  1996  by  Leo  J.  Salemann  of  Lockheed  Martin  Information  Systems,  Bellevue, 
WA.  This  database,  also  constructed  from  the  S-1000  source  file,  is  commonly 
displayed  on  the  “Stealth  Viewer”  using  the  GTIOO  graphics  engine. 
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Figure  1.  Hunter-Liggett  Terrain  Database  Schematic  Showing  Reflected  Area 
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Figure  2.  Hunter-Liggett  Terrain  Database  (50  km  x  50  km) 
with  Rectangle  (20  km  x  10  km)  Selected  for  Reflection 
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Figure  3.  Symmetric  Terrain  Database  (20  km  x  20  km) 
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III.  ANALYSES 


The  resources  available  for  this  task  allowed  a  small  series  of  experiments.  The 
goals  in  selecting  the  analytic  issues  and  defining  the  experimental  parameters  were,  first, 
to  test  the  new  database  and  illustrate  ways  in  which  SA  might  be  used  and,  second,  to 
produce  useful  results.  Exploring  some  aspects  of  ModSAF  behavior  and  determining 
factors  within  the  simulation  that  affected  battle  outcomes  was  judged  to  fit  these  criteria. 

ModSAF  is  the  most  widely  used  DIS  simulator  of  computer-generated  forces 
because,  by  using  it,  a  single  operator  can  manage  the  simulation  of  numerous  vehicles. 
The  operator  gives  instructions  concerning  force  size  and  characteristics  and  commands 
the  forces  by  issuing  orders  such  as  the  paths  they  should  follow  and  engagement  orders. 
ModSAF  itself  controls  actual  vehicle  movement  and  firing  decisions.  The  algorithms  it 
uses  are  intended  to  conform  to  vehicle  characteristics  and  military  practice,  but  the 
complexity  of  behavior  in  a  battlefield  environment  clearly  presents  a  simulation 
challenge.  Operators  at  IDA  and  elsewhere  have  observed  unexpected  behaviors  and 
battle  outcomes.  SA  is  an  ideal  tool  for  controlling  parameter  variations  and  isolating 
factors  that  influence  the  results. 

Rather  than  being  completely  planned  in  advance,  the  experiments  were  largely 
exploratory,  with  results  from  one  set  of  runs  suggesting  further  tests.  Except  for  the 
force  ratio  tests,  all  tests  used  identical  Ml  tank  forces  on  each  side,  typically  with  14  or 
16  vehicles.  The  experiments  were 

•  Terrain  and  Motion  Dependencies— ^hese  exercises  were  conducted  on 
standard  (asymmetric)  terrain,  with  a  force  on  an  open  plain  attacking  a  force 
on  ridges  and  hills,  to  determine  which  force  had  an  advantage.  Previous 
ModSAF  experience  indicated  that  the  attacking  force  would  have  an 
advantage,  which  contradicts  commonly  accepted  military  tactical  concepts. 

•  Force  Ratio  Analysis — These  exercises  were  symmetric  except  for  varying  the 
size  of  the  opposing  forces  to  examine  how  losses  varied  with  force  ratio. 

•  Synchronized  Attack — ^These  exercises  had  a  force  conducting  a  pincer  attack 
against  the  flanks  of  a  force  that  thought  it  was  moving  to  frontal  contact. 
Each  force  was  internally  symmetric,  with  the  symmetry  used  to  precisely 
coordinate  the  movement  of  the  pincers.  In  a  variation,  the  attack  was  from 
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two  sides  but  the  defenders  were  arranged  to  face  each  attack.  The  objective 
was  to  examine  the  value  of  the  coordinated  attack. 

•  Hull  and  Turret  Orientation — These  exercises  were  variations  of  an  attack 
from  two  sides  with  the  defenders’  tank  hulls  or  turrets  not  facing  the  attack. 
The  object  was  to  determine  the  effects  of  orientation. 

•  Other  Investigations — These  exercises  examined  the  relative  performance  of 
moving  and  stationary  forces  and  the  impact  that  firing  the  first  shot  of  an 
engagement  had  on  the  outcome. 

A.  TERRAIN  AND  MOTION  DEPENDENCIES 

In  previous  experiences  with  ModSAF,  study  team  members  observed  the 
unexpected  results  that  an  armored  force  moving  across  a  plain  and  attacking  a  stationary 
defensive  force  in  hilltop  positions  seemed  to  have  an  advantage. 

A  series  of  initial  runs  were  made  using  asymmetric  terrain,  and  equal  forces  of 
Ml  tanks  with  one  force  placed  on  ridges  and  hilltops  and  the  other  force  attacking  across 
an  open  flat  area.  Repeated  trials  confirmed  that  the  attackers  won  consistently. 

Next,  defensive  positions  were  carefully  modified  in  an  attempt  to  take  better 
advantage  of  the  terrain.  The  goal  was  to  place  the  defenders  slightly  behind  hill  crests, 
so  that  they  could  see  over  the  hilltops  but  were  partially  masked.  Repeated  trials  were 
made  with  continuing  slight  position  adjustments,  until  a  set  of  positions  was  found  that 
gave  the  defenders  the  advantage. 

These  trials  were  intended  for  learning  and  for  qualitative  assessment,  and  no 
numerical  results  are  presented  here.  Some  observations  include 

•  The  digital  terrain  database  essentially  consists  of  triangular  flat  areas  pieced 
together.  The  database  resolution  affects  the  characteristics  of  protected 
positions  (their  number,  locations,  degree  of  protection,  limits  on 
intervisibility,  etc.).  In  particular,  the  horizontal  resolution  for  the  Hunter- 
Liggett  terrain  data  of  about  125  meters  and  the  relatively  sparse 
representation  of  vegetation  severely  limit  the  terrain  advantage  of  defensive 
positions. 

•  Finding  protected  positions  using  the  ModSAF  displays  and  tools  is 
sufficiently  difficult  and  time  consuming  that  the  authors  believe  it  is  not 
possible  in  a  dynamic,  live  exercise.  In  the  specific  case  examined,  this 
represents  a  difference  between  ModSAF  and  either  manned  simulators  (in 
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which  each  tank  has  a  crew  that  can  position  it  using  a  3-D  view)  or  actual 
tanks.  And  this  difference  significantly  affects  the  battle  outcome. 

•  Presumably,  tanks  in  partially  masked  positions  are  harder  to  detect  and  hit. 
Other  factors  give  the  attackers  the  advantage  when  the  defenders  are  in  the 
open  on  hill  faces: 

-  Moving  Target  vs.  Moving  Shooter — ^Accuracy  is  degraded  when  the 
shooter  is  moving  or  when  the  target  is  moving.  The  formulas  used  in 
ModSAF  give  a  phit  advantage  to  a  moving  tank  shooting  at  a  stationary 
target  over  a  stationary  tank  shooting  at  a  moving  target. 

-  Exposed  Area — Tanks  on  an  open  slope  may  expose  a  significant  part  of 
their  top  surface  to  tanks  on  the  plain. 

B.  FORCE-LEVEL  ANALYSIS 

The  objective  of  this  analysis  was  to  determine  how  initial  differences  in  force 
levels  affected  the  outcome  of  a  battle.  Four  sets  of  runs  were  made  in  a  single  terrain 
area  with  10  runs  in  each  set  and  force  ratios  ranging  from  1:1  to  2.33:1.  In  each  set  of 
runs,  the  Blue  force  consisted  of  an  armor  company  (14  Ml  tanks).  The  Red  force  varied, 
with  the  different  sets  having  14,  10,  8,  and  6  Mis.  The  results  are  summarized  in  Table  1 
and  Figure  1 . 


Table  1.  Average  Losses  with  Varying  Force  Levels 


Number  of 

Red  Vehicles 

Mean 

Blue  Losses 

Mean 

Red  Losses 

6 

2.4 

6.0 

8 

4.1 

7.9 

10 

7.1 

9.2 

14 

9.1 

10.8 

These  results  are  generally  consistent  with  accepted  knowledge  (e.g.,  as  predicted 
by  the  Lancaster  equations)  that  favorable  force  ratios  yield  increasingly  better  outcomes. 
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Figure  4.  Loss  Exchange  Ratio  as  a  Function  of  initial  Force  Ratio 

C.  SYNCHRONIZED  ATTACK 

Two  sets  of  16  runs  each  were  made  exploring  the  value  of  synchronized  attacks 
from  multiple  directions.  Figure  2  diagrams  the  two  test  scenarios.  Each  scenario  used  4 
platoons  of  4  Mis,  totaling  a  company  of  16  Mis,  on  each  side.  The  forces  for  each  side 
were  internally  symmetric  but  not  symmetric  to  the  forces  on  the  other  side.  In  scenario 
1,  the  four  Red  platoons  move  toward  a  hill  occupied  by  Blue,  and  two  of  the  Blue 
platoons  come  down  the  hill  to  meet  them.  Shortly  after  these  forces  engage,  the 
remaining  two  Blue  platoons  conduct  a  coordinated  pincer  attack  on  Red’s  rear  flanks. 
Each  run  was  stopped  after  the  last  of  the  Red  forces  crossed  a  line  near  the  hill.  Scenario 
2  used  relatively  flat  terrain  and  all  forces  were  stationary,  with  each  platoon  facing  an 
enemy  platoon.  Runs  were  stopped  after  a  set  time  (3  minutes). 

The  entering  hypothesis  for  each  scenario  was  that  Blue  should  win  most  often. 
In  scenario  1,  the  Blue  pincer  units  attack  broadside,  thus  having  a  greater  and  more 
vulnerable  target  cross  section  to  hit.  Secondly,  the  Blue  force  is  dispersed,  with  each  gun 
pointed  initially  in  the  general  direction  of  the  entire  Red  force,  while  the  Red  force  is 
relatively  concentrated  with  each  gun  initially  pointed  only  at  part  of  the  Blue  force.  This 
could  mean  that  a  Red  tank  would  have  more  difficulty  finding  targets  and  also  would 
have  to  rotate  its  turret  farther  to  engage  targets  (particularly  after  a  number  have  been 
killed).  In  scenario  2,  only  the  second  factor  applies. 
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Figure  5.  Synchronized  Attack  Diagrams 

Table  2  summarizes  the  test  results.  Blue  did  better  in  scenario  1,  but  the  sides 
were  nearly  equal  in  scenario  2. 


Table  2.  Synchronized  Attack  Results 
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D.  HULL  AND  TURRET  ORIENTATION 

The  synchronized  attack  results  presented  in  the  previous  section  indicated  that 
the  Blue  force  had  a  significant  advantage  in  its  pincer  attack.  The  scenario  2  results 
indicated  the  advantage  did  not  derive  from  the  dispersed  disposition  of  the  Blue  forces. 
However,  in  each  scenario  the  initial  Blue  forces  encountered  by  Red  were  directly  in 
front  of  them,  so  that  any  impact  of  having  to  rotate  a  turret  to  engage  a  target  might  be 
obscured. 

Scenario  2  was  used  as  a  base  case  for  further  tests,  with  new  sets  of  runs  pointing 
the  Red  tanks’  hulls,  or  the  turrets,  or  both,  at  a  90°  angle  to  Blue.  Each  set  consisted  of 
1 6  runs,  except  that  24  runs  were  made  in  the  set  with  hulls  at  90°  and  turrets  pointed  at 
Blue.  The  variations  are  illustrated  in  Figure  3,  and  the  resulting  mean  differences 
between  Blue  and  Red  losses  are  summarized  in  Table  3. 
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13-  -■■-  -E3 

C3-  -1  1-  ^ 
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B-  -Q 

ii  ^ 
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Turrets  90  o 
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Figure  6.  Hull  and  Turret  Orientations 
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Table  3.  Mean  Excess  Red  Losses 
for  Different  Orientations 


Hull  Angle 

Turret  Angle 

0° 

90° 

-0.19 

1.69 

90° 

2.96 

3.88 

Table  3  shows  that  Red  losses  increase  when  either  the  hulls  or  turrets  are  not 
pointed  towards  Blue,  with  the  hull  direction  having  the  larger  effect.  This  is  a  2x2 
factorial  experiment,  with  2  levels  (0”  and  90°)  of  each  of  the  two  factors  (hull  angle  and 
turret  angle).  An  analysis  of  variance  shows  the  significance  of  the  hull  orientation  below 
the  2  percent  level,  while  the  turret  orientation  effect  is  significant  only  at  the  1 8  percent 
level  (an  effect  this  large  would  arise  from  random  variations  about  18  percent  of  the 
time).  There  was  no  significant  interaction  between  the  two  effects;  that  is,  within 
differences  due  to  random  variations,  rotating  the  Red  turrets  increases  the  Red  losses  by 
about  the  same  amount,  whether  or  not  the  hulls  are  rotated. 

E.  OTHER  INVESTIGATIONS 

The  relative  performance  of  moving  and  stationary  vehicles  was  mentioned  in 
Section  A  above  but  was  not  tested.  Another  factor  of  potential  significance  noted  in 
conducting  previously  described  tests  is  which  side  shoots  first.  This  section  describes 
investigations  into  these  factors. 

The  first  set  of  20  runs  investigated  moving  versus  stationaiy  vehicles  and  was 
designed  to  be  as  symmetric  as  possible.  Each  force  was  given  the  same  fire  permission 
range  (2  km),  and,  at  the  time  the  moving  force  closed  to  this  range,  the  positions  were 
symmetric,  in  symmetric  terrain  (see  Figure  4).  Each  side  had  14  Ml  tanks,  and  each  trial 
continued  until  the  forces  closed  to  0.5  km. 
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Figure  7.  Moving  vs.  Stationary  Vehicles 


Table  4  summarizes  the  results  of  these  trials.  The  moving  force  suffered  fewer 
average  losses  and  won  most  of  the  trials  (where  winning  is  defined  as  having  fewer 
losses),  and  also  shot  first  in  19  of  the  20  trials.  Using  a  paired  comparison  (t-test),  the 
differences  in  losses  between  the  sides  with  stationary  and  moving  vehicles  was 
significant  at  the  12  percent  level.  However,  using  the  same  test,  the  differences  in  losses 
between  the  side  that  shot  first  and  the  other  side  was  significant  at  the  7.5  percent  level. 
The  initial  hypothesis  was  that  different  outcomes  would  be  caused  by  differences  in  the 
formulas  used  to  compute  hit  probabilities  for  moving  vehicles  shooting  at  stationary 
targets  and  stationary  vehicles  shooting  at  moving  targets.  However,  the  results  introduce 
the  possibility  that  the  advantage  derives  at  least  in  part  from  the  fact  that,  in  these  runs, 
the  moving  vehicles  tend  to  shoot  first,  for  unknown  reasons. 


Table  4.  Moving  vs.  Stationary  and  First  Shooter  vs.  Not  First  Shooter 


Average  Losses 

Winner 

(Number  of  Runs) 

Stationary 

Moving 

Difference 

Moving 

stationary 

6.8 

5.3 

1.5 

11 

9 

1®*  Shooter 

Not  1®*  Shooter 

Difference 

1®*  Shooter 

Not  1®*  Shooter 

6.9 

5.2 

1.7 

12 

8 
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Further  tests  were  conducted  to  examine  the  impact  of  shooting  first.  These  were 
symmetric  tests  with  14  stationary  Mis  on  each  side  facing  each  other  2  km  apart,  as 
shown  in  Figure  5.  Sixteen  runs  were  made,  with  Blue  shooting  first  in  seven.  Red 
shooting  first  in  eight,  and  essentially  simultaneous  first  firing  in  one.  In  the  trials  in 
which  Blue  shot  first,  it  averaged  0.28  fewer  losses  than  Red.  However,  in  the  trials 
where  Red  shot  first,  it  averaged  1.5  more  losses  than  Blue.  Overall,  the  first  shooter 
averaged  0.67  more  losses  than  the  second. 

These  results  indicate  that,  in  the  tests  of  moving  versus  stationary  vehicles,  the 
advantage  for  the  moving  vehicles  was  not  due  simply  and  directly  to  their  frequency  of 
shooting  first.  The  greater  probability  of  hit  for  a  moving  vehicle  against  a  stationary 
target  compared  to  a  stationaiy  vehicle  against  a  moving  target  may  account  for  much  of 
the  observed  advantage.  However,  the  consistent  pattern  of  the  moving  side  shooting 
first,  and  its  strong  correlation  to  the  moving  side’s  loss  advantage,  indicate  that  other 
factors  may  be  significant.  It  may  be,  for  example,  in  the  moving  versus  stationary  trials, 
that  most  of  the  moving  vehicles  shot  before  any  of  the  stationary  vehicles,  but  in  the  later 
stationary-vehicle  trials,  there  was  a  more  even  mix. 


Figure  8.  Stationary,  Symmetric  Force 


The  last  series  of  tests  attempted  to  examine  why  one  side  might  tend  to  shoot 
consistently.  Only  two  tanks  were  used  in  these  tests,  one  stationary  and  one  moving,  as 
shown  in  Figure  6.  Ten  runs  were  made  in  which  the  stationary  (Blue)  vehicle  fired  first 
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nine  times,  and  the  two  vehicles  fired  nearly  simultaneously  in  the  other.  This  is  in 
contrast  to  the  earlier  results  in  which  the  moving  vehicles  tended  to  fire  first. 


Blue 

(Stationary) 


Red 

]  (Moving) 

0.5  km 


axis  of  symmetry 


Figure  9.  Moving  vs.  Stationary,  Two  Vehicles 


Since  only  two  vehicles  were  involved  in  these  tests,  it  was  fairly  easy  to  see  why 
the  stationary  tank  fired  first.  As  the  Red  tank  started  moving,  its  turret  would  swing 
from  side  to  side  over  a  wide  (-135°)  arc.  The  precession  was  essentially  the  same  in 
each  run,  and,  at  the  time  the  tanks  reached  their  engagement  range  (1.7  km),  its  turret 
would  be  pointed  well  away  from  Blue.  Blue’s  turret  also  processed,  but  at  the 
engagement  range  its  turret  would  be  pointed  approximately  at  Red.  Six  additional  runs 
were  made  with  the  engagement  range  changed  to  2.1  km,  which  led  to  a  geometiy  in 
which,  at  the  engagement  range,  each  vehicle’s  turret  pointed  approximately  at  the  other. 
Each  vehicle  fired  first  in  three  of  the  six  runs. 

F.  PRELIMINARY  FINDINGS 

The  puipose  of  this  chapter  is  to  illustrate  the  use  of  symmetric  terrain  and 
symmetric  analysis.  The  most  economical  way  to  do  this  was  to  examine  the  “black-box” 
behavior  of  the  ModSAF  simulation.  Symmetric  terrain  and  symmetric  analysis  were 
consistently  useful  in  setting  up  controlled  experiments  that  isolated  the  effect  of 
individual  variables.  Although  very  limited  in  scope,  the  tests  that  were  conducted  allow 
the  formulation  of  some  preliminary  findings,  as  well  as  some  unresolved  issues,  that  may 
be  worth  further  investigation. 

Several  results  are  consistent  with  expectations: 
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•  The  effect  of  initial  force  ratios  on  the  loss  ratio  is  generally  consistent  with 
expectations  based  on  Lanchester  equations. 

•  The  effect  of  vehicle  aspect  is  unsurprising.  It  is  significantly  disadvantageous 
to  engage  the  front  of  enemy  tanks  while  they  are  firing  at  the  side  of  your 
tank. 

•  Initial  relative  turret  orientation  has  a  smaller  effect,  but  the  effect  (in 
ModSAF)  may  be  real. 

The  results  of  the  first  set  of  moving  stationary  trials  seemed  anomalous  in  at  least 
one  respect.  The  moving  side  shot  first  in  19  of  the  20  runs.  This  result  suggests  that,  at 
least  with  respect  to  shooting  first,  the  trials  were  not  independent.  The  subsequent  one- 
on-one  trials  suggest  that  shooting  first  may  be  a  function  of  the  arbitrary,  but  largely 
deterministic,  precession  phases  of  the  turrets.  However,  we  did  not  do  additional  runs  to 
rule  out  a  positive  correlation  between  movement  and  shooting  first. 

In  terms  of  outcomes  on  these  20  runs,  the  moving  side  had  a  small  advantage  and 
the  side  shooting  first  had  a  somewhat  larger  advantage.  We  peeked  into  the  black  box 
and  determined  that  ModSAF  gives  a  probability  of  hit  advantage  to  movers  in 
engagements  with  stationary  targets.  In  and  of  itself,  shooting  first  may  not  convey  a 
tactical  advantage  unless  the  relative  precession  phases  strongly  favor  the  side  that  shoots 
first.  More  runs  would  be  needed  to  resolve  these  possibilities  or  develop  other  possible 
explanations.  The  foregoing  interpretations  do  not  consider  the  possibility  that  even 
when  the  forces  are  identical  on  both  sides,  ModSAF  algorithms  for  friendly  and  enemy 
forces  may  be  different.  While  we  have  no  reason  to  believe  this,  it  is  certainly 
something  that  should  be  ascertained  (by  symmetric  analysis?)  in  any  further  ModSAF 
investigation. 

Finally,  it  is  clear  that,  with  more  resources,  S5anmetric  analysis  could  be  used  in 
man-in-the-loop  evaluation  of  C  systems.  It  could  be  also  be  used  to  evaluate  weapons 
and  other  systems. 
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